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ly-C ! We investigate the experimentally allowed parameter space of an extension of the standard 

O I model (SMS) by one additional family of fermions. Therefore we extend our previous study of 

' the CKM like mixing constraints of a fourth generation of quarks. In addition to the bounds 

from tree-level determinations of the 3x3 CKM elements and FCNC processes {K-, D-, Bd-, 
Bg-mixing and the decay b — )■ S7) we also investigate the electroweak S, T, U parameters, the 
angle 7 of the unitarity triangle and the rare decay Bg — )■ ix^ n~ . Moreover we improve our 
5^ \ treatment of the QCD corrections compared to our previous analysis. We also take leptonic 

contributions into account, but we neglect the mixing among leptons. As a result we find that 
typically small mixing with the fourth family is favored, but still some sizeable deviations 
from the SM3 results are not yet excluded. The minimal possible value of Vth is 0.93. Also 
very large CP-violating effects in B^ mixing seem to be impossible within an extension of 
the SM3 that consists of an additional fermion family alone. We find a delicate interplay 
of electroweak and flavor observables, which strongly suggests that a separate treatment of 
the two sectors is not feasible. In particular we show that the inclusion of the full CKM 
dependence of the S and T parameters in principle allows the existence of a degenerate 
fourth generation of quarks. 
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1 Introduction 

Increasing the number of fermion generations (see [1] for a review and [2] for an update) 
is probably the most obvious extension of the usual standard model with three generations 
(SMS). Although being popular in the 80s, such a possibility was discarded for a long time. 
Recently these models (SM4) celebrated a kind of resurrection. Partly, this was due to the fact 
that a fourth generation is not necessarily in conflict with electroweak precision observables 

Besides being a straightforward extension of the SMS, an increase of the number of fermion 
generations leads also to several desired effects: 

• The authors of [71 [9l [HI [12], [TSj [I5] have shown that a fourth generation softens the 
current low Higgs mass bounds from electroweak precision observables, see e.g. [16], by 
allowing considerably higher values for the Higgs mass. 

• It might solve problems related to baryogenesis: An additional particle family could 
lead to a sizeable increase of the measure of CP- violation, see [IT] [18]. Moreover, 
such an extension of the SM would increase the strength of the phase transition, see 
[191 [201 [21]. 



• The gauge couplings can in principle be unified without invoking SUSY 

• New heavy fermions lead to new interesting effects due to their large Yukawa couplings, 
see e.g [23l[2^ . Moreover dynamical electroweak symmetry breaking might be triggered 
by these heavy new fermions [25l[26l[27l[28l[29l[3ni[3ll[32l[33l[34l[35]. This mechanism 
can also be incorporated in models with warped extra dimensions, as done in [36l [37] . 

There are also some modest experimental deviations that could be explained by the existence 
of a fourth generation: 

• A new family might cure certain problems in flavor physics (CP-violation in S^-mixing, 
K — TT-puzzle, e/c anomaly,...) see e.g. [381 [33 [iQl [lH [12] for some recent work and e.g. 
[^ [13] for some early work on 4th generation effects on flavor physics. 



• Investigations of lepton universality show a value of the PMNS element VeA 7^ at the 
2.5 a level [15]. 

For more arguments in favor of a fourth generation see e.g. [2] and also [ISl [HI [IHl [IS]. 
We conclude the list by repeating our statement from [5U]: In view of the (re)start of the 
LHC, it is important not to exclude any possibility for new physics scenarios simply due to 
prejudices. Direct search strategies for heavy quarks at the LHC are worked out e.g. in 
[3T1 [5T| [52| [53| [5^ [55| [56| [57] . Signatures and consequences for collider physics, such as the 
modification of production rates, have been studied e.g. in [5H1 [311 [HHl [GDI [61] . 

In this work we extend our analysis in [50], where we performed an exploratory study 
of the allowed parameter range for the CKM like mixing of hypothetical quarks of a fourth 
generation. Adding one generation of quarks results in several new parameters. In particular, 
we have the new masses my and mf, and nine parameters (six angles and three phases) in 
the 4x4 CKM matrix (compared to three angles and one phase in the SM3). Following 
our previous strategy, we consecutively add bounds on the CKM structure of the SM4 and 
perform a scan though the parameter space of the model to identify the allowed regions; 
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while this treatment is insufficient to fit for the central values or standard deviation of the 
model parameters, it gives a very reasonable idea of the experimentally possible parameter 
space allowing for statements on the size of effects of the model on particularly interesting 
flavor observables. Apart from the unitarity of the 4x4 matrix and the direct bounds on 
the quark masses, the most important input comes from direct measurements of the absolute 
values of CKM matrix elements, e.g. from /3 decay. In [50] the next step was the inclusion 
of flavor observables sensitive to FCNC, mediated i.e. by box or penguin diagrams. This led 
to some surprising results regarding the possible size of the quark mixing with the fourth 
generation quark, as rather large values for the mixing angle S34 could not be excluded. 

However, in |62] Chanowitz found that the parameter sets, which we gave as an example 
for large mixing with the fourth generation, are excluded by electroweak precision constraints, 
in particular by the oblique corrections [63]. Moreover, Chanowitz performed the whole 
electroweak fit for four different values of the mass of the t' quark. Here, some assumptions 
were used: i) the lepton masses are fixed to 771^4 = 145 GeV and rriy^ = 100 GeV, ii) 
lepton mixing is not included, iii) the mass difference of the heavy quark doublet is fixed 
to rrit' — my = 55 GeV, iv) only mixing between the third and fourth family was included. 
Assumptions iii) and iv) were also tested in [62]. 

Therefore, we supplement the analysis of the flavor sector by the S, T and U parame- 
ters; also the lepton masses of the fourth generation have to be taken into account. For the 
present work we assume that the neutrinos have Dirac character and neglect the possible 
mixing of the fourth neutrino in the lepton sector. Moreover, we extend the set of our FCNC 
observables to include also Bg — )■ /x^/x~ and we improve the simplified treatment of the decay 
b — )■ s'-f by using the full leading logarithmic result. Concerning the tree- level determination 
of the CKM elements we include now also the experimental results for the angle 7 of the 
unitarity triangle, which gives a direct constraint on CKM phases. Similar studies have been 
recently performed e.g. in [511 ESI ESI EZ] 

In Section 2 we present all experimental constraints we use in our analysis. We start with the 
parameterization of Vckma in Section 2.1, next we discuss briefly tree- level determinations 
of CKM elements and direct mass limits. The electroweak parameters S, T and U will be 
investigated before reviewing the FCNC constraints. We end Section 2 with the allowed 
regions for deviations of the SM4 results from the SMS values. 

In Section 3 we determine the bounds on the parameters of the model. After explaining our 
general strategy in 3.1, we present the results for the different mixing angles of Vckma, the 
new results for Vex and Vtx {x = d, s, b), and allowed effects of a fourth generation in neutral 
meson mixing. 

In Section 4 we give a Wolfenstein-like expansion of the 4x4 CKM matrix. With the addi- 
tional information from the electroweak sector, tighter constraints on the fourth generation 
quark mixing can be utilized leading to a simplified expansion. 
We conclude with Section 5. 



2 Constraints on Vckma 



2.1 Parameterization of Vck 



AM 



In the SM3 the mixing between quarks is described by the unitary 3x3 CKM matrix [68| 169] , 
which can be parameterized by three angles, 6*12, 6*13 and 623 {6ij describes the strength of 



the mixing between the ith and jth family) and the CP- violating phase 613. The so-called 
standard parameterization of Vckms reads 
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(2.1) 



with 

Sij := sin(^jj) and Cjj := cos(6'jj) . (2.2) 

Extending the minimal standard model to include a fourth family of fermions (SM4) intro- 
duces 3 additional angles in the CKM matrix ^14, ^24 and ^34 and 2 additional CP-violating 
phases Su and ^24. To determine the allowed range for these new parameters we use an exact 
parameterization of the 4x4 CKM matrix. We have chosen the one suggested by Botella 
and Chau |70p . Fritzsch and Plankl |71)i and also by Harari and Leurer 
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(2.3) 

For our strategy the explicit form of Vckma does not matter, it is only important that the 

parameterization is exact. Besides the nine parameters of Vckma we have also the masses of 

the fourth generation particles, which we denote as my^mti^mi^ and rrty^. We do not include 

leptonic mixing, yet. 



2.2 Experimental bounds 

In this section we summarize the experimental constraints that have to be fulfilled by the 
parameters of the fourth family. 

The elements of the 3x3 CKM matrix have been studied intensely for many years and 
precision data on most of them is available. In principle there are two different ways to 
determine the CKM elements. On the one hand, they enter charged weak decays already at 
tree-level and a measurement of e.g. the corresponding decay rate provides direct information 
on the CKM elements (see e.g. [73] and references therein). We will refer to such constraints 
as tree-level constraints. On the other hand, processes involving flavor- changing neutral 



^In the published paper of Botella and Chau there is a typo in the element Vtd'- in the last term of Vtd 
the factor Sy has to be replaced by Cy. 

^In the published paper of Fritzsch and Plankl there is a typo in the element Vcb'- the factor C23 has to be 
replaced by the factor S23. 



currents (FCNC) are forbidden at tree-level and only come into play at loop level via the 
renowned Penguin and Box diagrams. These processes provide strong bounds — referred to 
as FCNC constraints — on the structure of the CKM matrix and its elements as well as on 
the masses of the heavy virtual particles appearing in the loops. 

We will start with the tree-level constraints, since they only depend on the CKM elements and 
not on the fermion masses. Next we consider mass constraints on the fourth family members 
from direct searches at colliders. Since the oblique electroweak parameters are expected to 
reduce the allowed range of masses for a new fermion family notably, we consider them next 
and finally we discuss the FCNC constraints. 



2.2.1 Tree-level constraints for the CKM parameters 

Since the (absolute) value of only one CKM element enters the theoretical predictions for 
weak tree-level decays, no GIM mechanism or unitarity condition has to be assumed. By 
matching theory and experiment the matrix element can be extracted independently of the 
number of generations q Therefore, all tree-level constraints have the same impact on the 
4x4 matrix as they have on the 3x3 one. 

We take the PDG values [71] for our analysis: 





absolute value 


relative error 


direct measurement from 


Kd 


0.97418 ± 0.00027 


0.028% 


nuclear beta decay 


Vus 


0.2255 ±0.0019 


0.84% 


semi-leptonic K-decay 


Vub 


0.00393 ± 0.00036 


9.2% 


semi-leptonic B-decay 


Vcd 


0.230 ±0.011 


4.8% 


semi-leptonic D-decay 


Vcs 


1.04 ±0.06 


5.8% 


(semi-)leptonic D-decay 


Vcb 


0.0412 ±0.0011 


2.7% 


semi-leptonic B-decay 


Vt, 


>0.74 




(single) top-production 



In the following, we denote the absolute values in the table above as |Vi| ± AV^. In addition 
to the above tree-level constraints there exists a direct bound on the CKM angle 



7 = arg 






It can be extracted via the decays B — )■ DK, Dtt [751 CHI UH [78]. In principle the extraction 
of 7 might be affected by the presence of a fourth generation of fermions [79], but it was 
shown in [80] that these effects are negligible. Therefore, 7 gives direct information on the 
phases of the CKM matrix; with three CP violating phases present, this can provide a useful 
piece of information. We use the CKMfitter value from [81] (update of [821 183] ) 
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where the last error accounts for the tiny additional uncertainty due to the additional fermion 
generation. 



■^There is, however, one loop hole: In 05] the possibility of lepton mixing reducing the accuracy of the 
determination of e.g. Vud was discussed. Since we use the more conservative error estimate from the PDG, 
our relative error is similar to the final error of Lacker and Menzel, who started with a more ambitious error 
for Vud in their analysis [45]. 



2.2.2 Direct mass limits for the fourth family 

The PDG [74] gives from direct searches the following mass limits for a fourth family 

m^, > 80.5...101.5GeV, (2.5) 

m/, > 100.8 GeV, (2.6) 

my > 128. ..268 GeV, (2.7) 

mt' > 256 GeV. (2.8) 

The mass bound on the heavy neutrino depends on the type of neutrino (Dirac or Majorana) 
and whether one considers a coupling of the heavy neutrino to e~, fi~ or r~. It is interesting 
to note that LEP results in combination with |g^ exclude a fourth stable neutrino with m 
< 2400 GeV [71]. The quark mass bounds are obtained from direct searches at TeVatron 
[85| [86] . which were recently updated [8j 



rrih' > 338 GeV, rrif > 335 GeV. (2.9) 

In [89] it was pointed out that in deriving these bounds assumptions about the couplings 
of the fourth generation have been made (in [87] it is e.g. explicitly assumed that the 
b' is shortlived and that it decays exclusively to tW~, which corresponds to demanding 
Vub' ~ ~ Vcb', rny < rriti and Vty is not extremely small). Without these assumptions the 
mass bounds can be weaker, as the extraction of the masses has to be combined with the 
extraction of the CKM couplings. The inclusion of this dependence is beyond the scope of 
the current work. For some recent papers concerning the mass exctraction of leptons and 
quarks, see [HHl E] • 

In this work we investigate heavy quark masses in the range of 280 GeV to 650 GeV heavy 
charged lepton masses in the range of 100 GeV to 650 GeV and heavy neutrino masses in the 
range of 90 GeV to 650 GeV. Note that the triviality bound from unitarity of the t't' 5- wave 
scattering [92] indicates a maximal t' mass of around 504 GeV [93]. However, this estimate 
is based on tree-level expressions and while it seems prudent to treat too high quark masses 
with a grain of salt, one should not disregard higher masses based on this estimate alone. 
In this context it would be desireable to have e.g. a lattice study of the effect of very heavy 
(fourth generation) quarks. 

2.2.3 Electroweak constraints 

We present here the expressions for the oblique electroweak S, T and U parameters [63] in 
the presence of a fourth generation. They were originally defined as 



a^ = 4e^^[n33(g^)-n3Q(g^)] 



(2.10) 
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t/ = 4e2A[nn(g^)-n33(a] , (2.12) 
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with the electric couphng a and e, Vi.r^y denotes the virtual self-energy contributions to the 
weak gauge bosons and with the Weinberg angle expressed as x-w = sin^ 9]^ and xw = l—x^. 
In the first paper of Ref. [63] xi^Mf was approximated by M^. The T parameter is related 
to the famous p-parameter [Ml [951 192] 
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In practice, it turns out to be considerably simpler to reexpress the derivatives in 5" and U 
as differences 
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This approximation works very well for mnew ^ Mz and it is used by the PDG [71]. We will 
use however the original definitions given in Eqs. (I2.10p . (12. lip . (I2.12p with M^ = Xu;M|, 
because there are no correction terms and our expressions are exact. 

Next only the new physics contributions to the S, T and U parameters will be considered, 
as the SM values of the oblique parameters are by definition set to zero. Fit results for the 
allowed regions of the S, T and U parameters are obtained e.g. by the PDG [71], EWWG 
[96] , Gfitter [16] and most recent in [H]. Note that the more recent analyses [161 HI] differ 
significantly from the old (November 2007) PDG version. Due to more refined experimental 
results and an improved theoretical understanding the best fit values shifted significantly 
towards higher values of S and T, see Fig. [T]for the Gfitter S-T ellipse [HZ]; this somewhat 
relaxes the previously observed tension with an additional fermion generation. 
In the presence of a fourth generation the fermionic contribution to these parameters (before 
the necessary subtraction of the SM contribution) reads 
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Uf denotes the up-type quark of the /th generation, df the down-type quark of the /th 
generation. If the charged lepton of the /th generation and Uf the neutrino of the /th 
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Figure 1: Fit of the electroweak oblique parameters S and T. The plot is taken from [97] . 



generation. We have used the following functions 
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Both functions are symmetric in their arguments. 

The formula for S is very well known - see e.g. [631 El IHl |62]. Using instead the PDG 
definition we would obtain the following corrections terms to S for heavy quark masses 
{ml > Ml) 
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which are very small for the allowed mass ranges of the fourth family members. In the param- 
eter T no mixing was usually assumed. We give here the full CKM and PMNS dependence. 
Our expression for T in Eq. (I2.18P agrees with the one quoted in [62] , if we make the same 
assumptions (only 4 — 3 mixing or 4 — 3 and 4 — 2 mixing is considered) . 
By defining the SM3 values for S and T as zero, we only need to take the additional con- 
tributions due to the fourth generation into account. Keeping the full, previously neglected 



CKM dependences, we obtain 
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5*4 has a large positive contribution of about 0.21 which is independent of the parameters 
(masses and mixing) of the model. This value can, however, be diminished by the second 
logarithmic term that depends on the fermion masses. 
In the SMS the only significant contribution to T reads 



Nr 



IGnXwXwM'^ 



[m'^^- Ft {ml ml)] . 



(2.27) 



Here safely Vtt = 1 can be assumed. In the SM4, however, Vtt can in principle differ signifi- 
cantly from one, therefore we have the correction term in "+Ft [m, 
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in the formula for 

T4. We also have included all previously neglected mixing terms within the SMS particles. 
In principle we also should correct for the charm-strange contribution and for the up-down 
contribution with "-|-Ft {m,'^,m,D + Ft (m^,??!^)" , but their numerical effect is considerably 
below one per mille of the top-bottom contribution, so we do not show these two additional 
correction terms in the formula for T4. 

With the help of the S and T parameter Chanowitz [62] could exclude the three parameter 
sets, which we gave in [50] as an example for a very large mixing between the third and the 
fourth generation; these sets have passed all bounds set by precision flavor observables. We 
confirm the numbers from table I in [62] . We also tested the approximation of taking only 
S — 4 mixing into account: Comparing with the full CKM dependence the differences are 
below 6 % for these three parameter sets. 
To simplify the expression for U we approximated 
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Moreover we have only shown the contributions of the 4th family explicitly in Eq. (12 .26 p . the 
previously neglected rest is denoted by Usms- It will be interesting to see in a future analysis, 
whether the large logarithms in the lepton sector will lead to strong constraints on the PMNS- 
matrix. In the literature it is typically assumed that U is very small, see [5] for a notable 
exception. To our knowledge we incorporate for the first time the full CKM dependence in 
U. We find that arbitrary values for mixing and mass parameters could in principle generate 
values as large as 7.5 for t/|j. If one only takes into account mixing parameters that pass the 
tree-level flavor constraints still values of (9(0.1) seem to be possible; however, in this case 
we observe a simultaneous blow up of the T parameter. For T < 0.4, U does not exceed 0.06. 
Note that, while still small, this value is larger than the 0.02 effect expected without flavor 
mixing [TT] . 

At this point a few comments are appropriate: first, we would like to point out that our 
implementation of the S and T parameter is not "exact" from the SM4 point of view. In 
principle one would have to perform a full reanalysis of all electroweak data from the SM4 
perspective to fit the new values of S and T, as advocated for in [52]. This is, of course, 
beyond the scope of the present work and it is generally accepted that a large deviation of 
the oblique parameters from their SM values cannot be accommodated in models that do 
not introduce new particles coupling to fermions [71]. Secondly, we will henceforth neglect 
the effect of lepton mixing due to a non-trivial modification of the PMNS matrix — the off 
diagonal elements including the fourth neutrino are in any case required to be small, see [ISj. 



As a first step in our analysis, we only take the tree-level constraints on Vckma into account 
and investigate the parameter ranges that pass the S — T test at the 95% confidence levelj. 
The following values for the SM fit of the oblique parameters are used 



^bestfit = 0.03 , as = 0.09 , (2.29) 

Tbestfit = 0.07, aT = 0.08, (2.30) 

Pcorr = 0.867 , (2.31) 

where ax gives the standard deviation of x. The S and T parameters are not independent 
quantities; the strength of this correlation is given by pcorr- 

As our work focuses on the flavor aspects of the 4th generation scenario, a short comment 
on the famous S-T-ellipses seems to be in order. If the U parameter stays close to zero for 
some physics model it is feasible to set ?7 = and work with S and T alone. In this case the 
probability distribution of S and T reduces to a two-dimensional Gaussian distribution in 
the S-T plane centered on the best fit values. Due to the different a^ and due to the strong 
correlation the distribution is essentially squeezed and rotated. Hence, the "equiprobability" 
lines are no longer circles but ellipses. In fact the two dimensional case is somewhat special 
as the problem of finding the ellipse encircling an area corresponding to certain probability 
P can be solved analytically. The equation determining the contour for a given confidence 



^We did not check whether this is the largest possible value. 

^For the final investigation of the allowed parameter range of a fourth fermion family we will use the 99% 
confidence level. 

^Note that this fit is the most restrictive one currently available. Using instead the results from Gfitter a 
little more space is left for a fourth family. We simply decided to use the most recent numbers. 



level CL is then given by 



S 
T 



•Jbcst fit 
Thcst fit 



o'sO'tP o'to't J 



S 
T 



•J best fit 
2b est fit 



-21n(l-CL) . 



(2.32) 



Already at that stage we find some interesting results: 

1. S — T test with 'no leptons' + 'no Vckm^ 

We do not take into account leptons as well as mixing of the quarks. By neglecting the 
leptonic contribution one can, of course, not make any conclusions as to how restrictive 
the oblique parameters are. However, we still find it instructive to consider the effect 
of the various contributions in the S-T plane individually. The scatter plot shows the 
accessable region within the 95% CL ellipse of [Hj. We find, as expected that the masses 
of the fourth quark generation can not be degenerate if they fulfill the constraints from 
the oblique parameters. The necessary mass difference is of the order of 50 GeV as 
stated in [TT]. 



1^ S-T plane 
-r- no leptons 
I no CKM 




0.15 0.2 



2. S — T test with 'with leptons' + 'no Vckm' 

Next, we include the leptonic contributions (without lepton mixing) and still neglect 
CKM mixing. In this case also degenerate values of the quark masses of the fourth 
generation are in principle not excluded; however, this would require a significant mass 
gap in the lepton doublet to increase T (and preferably reduce S). 



10 



1L S-T plane 
—. with leptons 
I noCKM 




The dark green points indicate values not in conflict with a degeneracy of the quark 
masses of the fourth generation. 

3. S — T test with 'no leptons' + 'with Vckm' 

To study the "unperturbed" effect of a non trivial CKM structure of the fourth gener- 
ation, let us discard again the leptons for the moment. The modified CKM structure 
results in an increase of the T parameter without changing S, cf. Eqs. fl2.24p and f l2.25p . 



1^ S-T plane 
-p no leptons 
I with CKM 




In this scenario an increase of T can originate either from a quark mass splitting or 
from nonzero mixing of the fourth generation with the SM quarks. However, a mass 
splitting must also induce a tiny (logarithmical) contribution to S; so the central area 
which could not be reached in scenario '1' corresponds to nonzero mixing and tiny mass 
splittings. 

4. S — T test with 'with leptons' + 'with Vckm^ 

Here, we use the full expressions for S and T including both, leptons (without mixing) 
and CKM mixing. In this scenario we find a maximally allowed mass splitting of 



mj/ 



rrib'l < 80 GeV for quarks and |m^ 



m„ 



< 140 GeV for leptons. 



Note that this splitting was also observed by the Gfitter group [57]; their fits also show 
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a minimal required mass splitting as they do not take the possible effects of a nontrivial 
CKM structure into account. Due to the effects of quark mixing, we do not find a lower 
bound for the splitting. In fact, a simultaneous degeneracy of quark and lepton masses 
is not excluded, even though the S parameter favors larger t' and U masses. 



i ^ S-T plane 
. _p with leptons 
1 with CKM ^ 
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5. For completeness we also show the S-U plane using the exact expression for f/ and S. 
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Summarizing our investigation of the 5, T and \J parameters we get the following results: 

• U is not a priori small; only after the constraints on the quark mixing and the T 
parameter are used the maximal value for V is reduced below 0.06. 

• The quarks of a 4th generation can be degenerated without violating the 95% CL 
constraints from electroweak precision observableCl- However, this requires taking into 
account the effects of the non trivial flavor sector on T, i.e. mixing of the 4th generation 
fermions, or a sufficiently large mass splitting in the lepton sector. Note that, at first 
glance, this result seems to be in direct confiict with the standard statement that 
a degenerate fourth generation is excluded at the 6 a level [71] by virtue of the S 



^For the 99% CL this statement wih even hold stronger. 
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parameter. However, this statement always tacitly assumed a trivial CKM structure. 
The CKM factors in Eq. fl2.25p can lead to T > even if both lepton and quark masses 
are degenerate. However, we did not investigate the effect of the Z ^ bb vertex, which 
tends to favor small or no mixing. Still one can conclude that the situation for tiny 
mass splittings or even degenerate masses drastically improves once mixing is taken 
into account. 

Finally, we also have to (re) consider the contribution of the Higgs particle, since in the 
presence of a fourth family higher values of the Higgs mass may be possible [63]. The 
correction terms to the S, T and U parameters read 



S 



H 



T. 



H 



1 

127r 


In 


(117GeV)2' 




3 


Ml 



IQnxw (117GeV)2 



U, 



H 



0, 



(2.33) 

(2.34) 
(2.35) 



Using that form for the Higgs contributions we implicitly subtract the used value for the 
Higgs mass in the fit (117 GeV) and add "our" value. 

A heavier Higgs increases S and lowers T. Instead of adding 5*^ and Th to our values 
for S and T, we subtracted the Higgs contributions from the fit values to make the diagram 
easier to understand. So we shift the ellipse and not our data sets. We investigate the S and 
T parameters for three values of the Higgs mass: 117 GeV, 250 GeV and 500 GeV. 




0.05 0.1 0.15 0.Z 0.25 



The decrease in T is welcome, as it allows even bigger mass splitting (or alternatively larger 
mixing); however, the simultaneous increase of S due to the heavy Higgs completely seems 
to neutralize or even reverse this effect. Hence, as stated recently in [13] very large values for 
the mass of a SM-like Higgs are clearly not favored. However, for a 250 GeV Higgs scenario 
the origin (SM3) is outside the ellipse, whereas some SM4 points are inside and thus more 
likely. 
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Figure 2: Schematic diagrams for the necessary ingredients of the golden plated channel 
Bd -^ J/'^Kg-. B-mixing, Kaon mixing and the decay process itself. The left panel shows the 
tree-level and the right panel the penguin mediated decay. The dashed line represents any 
current capable of creating a J/\l/, e.g. two gluons. 

2.3 Flavor physics constraints — FCNC processes 

After addressing the electroweak bounds, we turn to the constraints imposed by precision 
observables of flavor physics involving a FCNC. One can hope to impose severe constraints 
on the model by utilizing information from such processes as it is well known that the 
weak interaction bypasses the Appelquist-Carazzone decoupling theorem [98]; thus, FCNC 
processes are very sensitive to contributions of new physics. 

However, the selection of flavor physics bounds on a hypothetical fourth family is a non 
trivial issue. The reason for this is the fact that some processes known for being theoretically 
or experimentally very clean, may in fact specifically require the SMS setup. Hence, it is 
always necessary to check, if a specific feature (of the SMS) crucial e.g. for the data analysis 
is preserved in the fourth generation extension. If this is not the case, it may either be 
necessary to repeat the analysis without some SMS simplification — much like the need to 
give up S X S unitarity — or the whole process may not even be feasible anymore. 

As an example for how unexpected complications may arise (see also [99] for a more 
detailed discussion), we discuss the so-called golden plated channel for the determination of 
the Standard Model CKM angle (3: B^ — ?■ J/'^Kg jlOOj . This channel is renowned for being 
theoretically very clean (in the SMS). Since the decay process is tree level dominated, it is 
usually taken for granted that the contribution of the fourth generation quarks to the decay 
is generally small. Therefore the SM4 could, in principle, be an explanation for discrepancies 
of the measurement of sin(2/3) in the Bd — )■ J /'^Kg and Bd — )■ ^Kg., as Bd — )■ ^Kg is penguin 
dominated and as such more sensitive to new physics effects, see |101] for a more detailed 
version of this argument. However, it turns out that this elegant picture of the consequences 
of the fourth generation is, unfortunately, too simple. The reason for this is the following: 
sin(2/3) is extracted via time-dependent CP asymmetries. The necessary ingredients are (i) Bd 
mixing, (ii) Kaon mixing and the (iii) decay process itself, see Fig. |2]for a schematic picture of 
the relevant subprocesses. There are in fact two decay processes, the tree-level decay and the 
top mediated penguin decay (c and u penguin are expected to be tiny). However, the beauty 
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of this process in the SMS is that the tree level decay and the t penguin have (to a fantastic 
accuracy) the same CKM phase. Hence, it is not necessary to take into account e.g. different 
hadronization effects as they will only modify the overall amplitude but not the phase. Adding 
an additional generation has two new, separate effects. First of all, the expressions for the box 
diagram changes (see formulae below), so that additional CKM factors contribute; therefore, 
instead of the CKM angle (3 a different combination of CKM angles can be extracted from 
this process. This is, however, not a problem as one could still use B^ — )• J/'^Kg to constrain 
the SM4. The real problem is the simultaneous modification of the penguin diagram by the 
t' loop. As the t' will introduce some new virtually unconstrained phase, penguin and tree 
decay now have different CKM phases; the fourth generation introduces a mismatch between 
tree and penguin decay, which makes taking into account hadronization and QCD corrections 
mandatory. Therefore, it is not clear what quantity can be extracted from time-dependent 
CP asymmetries in B^ — t- J /'^Kg in the SM4 scenario. This, of course, limits the usefulness 
of this process for constraining the parameters of the model. 

As the above example shows, not all processes can be used to obtain limits on the param- 
eters of the SM4 and one has to be careful not to make use of a bound whose experimental 
input essentially requires the SMS setup or some SMS specific feature. 

2.3.1 FCNC constraints with no sensitivity to lepton mixing 

Another issue is the sensitivity of some processes to the properties of the leptons. Since 
we do not take mixing in the lepton sector into account and in fact assume lepton number 
conservation, quantities that are insensitive to the precise structure of the lepton sector are 
of course advantageous. 

We first consider the mixing of the K-, the D-, the Bd- and the iJ^-system. For 
completeness we repeat the relevant formulae already given in [50] . The virtual part of the 
box diagram (here e.g. for i^^-mixing) 

b W d b u,c,t(,t') d 



->- 



r\/\/\/\/\^ 



u,c,t(,t') >' 



i'X/X/X/X/Xj 



> > s > s ►- 



u,c,t(,t') W? ?W 



d W b d u,c,t(,t') b 

is encoded in M12, which is very sensitive to new physics contributions. It is related to the 
mass difference of the heavy and light neutral mass eigenstate via 

/\M = Mbh - Mb^ = 2\Mi2\ . (2.S6) 
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In the SMS one obtains the following relations 



Mf oc 



Vcc (Af ) So{x,) + 2r],AfxfS{x,, x^) + Vtt [xf) So{xt) , (2.37) 

M^,'^ (X vu [xf'Y Soixt) , (2.38) 

Mg^ ^r]u{Xf'YSoixt), (2.39) 



with the Inami-Lim functions |102] 



^, , Ax-llx'^ + x^ 3x^lnb] /o .^n 



S{x,y) = xy 



1/13 1 3 1 



_y-x \A 21 -y 4(l-y)2 



Hy] 



1/13 1 3 1 \ , , , 3 1 1 

In X 



x-y\A 21-x 4(l-x)2y 4 1 - x 1 - y 



(2.41) 



where Xrt = tpt, the CKM elements 



Af = KdK*., Af'' = Kd^;, Af = V^sV:, (2.42) 

and the QCD corrections (TDSl [M [TDS] 

r/,, = 1.38 ±0.3, r/rf = 0.47 ± 0.04, r^^f = 0.5765 ± 0.0065. (2.43) 

The full expressions for M12 can be found e.g. in [103[ 1106] . In deriving these expressions 
the unitarity of the 3x3 matrix was explicitly used, i.e. 

A;f + Af + Af = . (2.44) 

Moreover, in the i?-system the CKM elements of the different internal quark contributions 
are all roughly of the same size. Only the top contribution, which has by far the largest value 
of the Inami-Lim functions, survives. This is not the case in the ii'-system. Here the top 
contribution is CKM suppressed, while the kinematically suppressed charm terms are CKM 
favored. Therefore, both have to be taken into account. More information about the mixing 
of neutral mesons can be found e.g. in |106[ 1107] . 

We define the parameter A as the ratio of the new physics model prediction (in our case 
SM4) for a generic observable to the SM3 theory value; thus, it quantifies the deviation from 
the standard model |106] . For M12 one would then define: 

A:=^^ = |A|e^<^". (2.45) 

This representation is convenient as one can effectively map any observable to the complex A 
plane; this allows a straightforward comparison of the sensitivity of the various observables 
to the effect of the model. Experimental data can be mapped analogously by plotting A : = 
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QExp jqsmz jj^ ^j^g same complex plane. O^^^ denotes the experimental value of a generic 
observable and O^^^ the theory prediction within the SMS. In the SM4, we obtain 



Mf'^^^ 



M,2 



Bd,SM4 



oc Vcc (Af ) So{x,) + 2r],tXfxfS{x,, Xt) + m (\f) So{xt) (2.46) 

+2r]ct'Xf X^ S{xc, Xf) + 2ritt'Xf X^ S{xt, Xf) + r]t>t' ix^ ) So{xt>) , 

oc r]u [xf'ySo{xt)+Vt't' [x^^fSoixf) + 2r]u'Xf''X^'S{xt,Xt') , (2.47) 

Mit'^*^^ (X rjtt (Xf^YSoixt) + rj,, {X^^Y Soix,) + 2r/«,AfA,^^5(xt, x,) . (2.48) 

Note that now also those CKM elements change that describe the mixing within the first 
three families! For simplicity we take the new QCD corrections to be (see also [66l [67] ) 

Vt't' = Vtt' = Vtt and r]ct' = ^ct ■ (2.49) 

It is interesting to note here that the only information we have currently about the CKM 
elements Vtd and Vts comes from i?-and i^-mixing plus assuming the unitarity of Vckm?.- 



The mass difference in the neutral D system can be used to infer a very strong bound 
on iKife'Kb'l, see [108)^1 . We redid this analysis in [50] and softened the bound. The mass 
difference in the neutral D°-system is typically expressed in terms of the parameter xr, : 

.,.^<^. p.50, 

J- D i- D 

For more information on the last inequality see e.g. the discussion in |110[llll] . HFAG |112] 
quotes for the experimental value of Xd 

xd = (0.811 ±0.334) -10-^ (2.51) 

The main difference compared to the above discussed K- and 5-mixing systems is that in 
the D-system the theory prediction in the SM3 is theoretically not well under control, see 
e.g. |110l llllj . However, the pure contribution of a heavy fourth generation to M12 can be 
calculated rehably. Using the unitarity of the CKM matrix of the SM4 Af° + Af°+Af VA^" = 
(with Af = VcxV*^ ), the full expression for M12 reads 

mS oc (Af )'5o(x,) + 2Af Af 5(x„x,) + (Af )' 5o(a:,) + LD 
+2Af Af 5(x„Xf,0 + 2Af Af 5(xb,XfeO + LD 
+ (Af)'5o(x6'), (2.52) 



^A similar strategy was recently used in |109| . 
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where the proportionahty constant is 



127r2 



f^Bnv{mc,Mw). (2.53) 



We use the same numerical values as in |50]. The first line of fl2.52p corresponds to the pure 
SMS contribution, the third line is due to contributions of the heavy 4th generation and the 
second line is a term arising when SMS- and b' contributions mix: 

mS = MgsMs + MSM^. + M,%, . (2.54) 

The idea of |lU8j was to neglect all terms in M^ , except M^ y , and to equate this term 
with the experimental number for xd, since all perturbative short-distance contributions with 
light internal quarks are negligible. Since it is not completely excluded that there might be 
large non-perturbative contributions to both M^ gj^j-^ and M(^ ^j^^ (denoted by LD)^ each of 
the size of the experimental value of xd-, we get the following bound 

3^D0,s.p > M^^y . (2.55) 



Allowing this possibility we obtain the following bounds on \Vub'Vc 



cb' 



( 0.00S95 for m^,, = 200 GeV , 
\Vub'Vcb'\ < I 0.00290 for m^,/ = SOOGeV, (2.56) 

[O.OOIQS for my = 500 GeV. 

This bound is still by far the strongest direct constraint on |Kib'K;b'|- 

Next, we consider the 6—7-87 transition. In [5U] we approximated the treatment 
of the FCNC decay 6 — )■ 57 by simply looking at the product of CKM structure and the 
corresponding Inami-Lim function D^^Xt) |102)l . 

_ \\l^'?D',{x,f + 2Re (Af^^Af ^^) D',{x,)D',M + \\m'D',M' ,. „. 



with 



r^l , ^ -7a; + 5x2 ^ g^3 ^2(^2 - Sx) , . , ,^ ^„, 



and Aj; = A^^ We assumed that parameters which give a value of l^h^s-^ close to one will 
also lead only to small deviations of r(6 — >■ s^')^^^'^ jYih — )■ 57)'^*'^^ from one. However, this 
crude treatment imposed a too strong bound on the S — 4 mixing. 

In this work we will use the full leading logarithmic expression for 6 — )■ 57, see also [66|l67]. 
Following |11S] we normalize the 6 — )■ 57 decay rate to the semi leptonic decay rate 



R:-- 



r(6^s7) \y:ytb? 6a 



Y{h^cev) \V,b? Tvfiz) 






2 



(2.59) 



f{z = ml/ml) is a phase space factor, which we will not need later on. It is interesting to 
note that in deriving this formula the unitarity of the S x S CKM matrix was already used 



^The Inami-Lim function D'^{xt) is proportional to the Wilson-coefBcient GT^{Myi/)- 
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and the CKM combination A„ = V*gVub was neglected (in comparison to Ac and A^). The 
effective Wilson coefficient C^ is a linear combination of the penguin Wilson coefficients 
C7 and Cg, which are accompanied by the CKM structure At and the current-current Wilson 
coefficient C2 with the corresponding CKM structures Ac and A^ 



^(0)e// 



(/^) 



Cf^' 



(/^) 



(jeff2 



(/^) 



(jeff3 



(/^) 



C^r"\fi)=V^^Ci'^{Mw) 



(2.60) 
(2.61) 



Cf^\f^) 



'4\Mo), 



v-^-v-nc^7iMw) 



(2.62) 



i=l 



with 



as{Mw) 



T]{fi) :-- 



asifJ.) 



(2.63) 



(2.64) 



The values for hi and aj are given in Table XXVII of |113] . The initial conditions of the 
Wilson coefficients read 






"2^0 I xt 



Ci'J{Mw) = --E',{x, 



mi 



mi 
Ml 



D'q{x) is given above in Eq. (I2.58p . Eq{x) reads 



-M^) 



2x + 5x — x^ 



3x' 



X] 



4(1 



Inbl 



(2.65) 
(2.66) 

(2.67) 



X] 



(2.68) 



Numerically it turns out that even for large values of mf (up to 1000 GeV) C^ is the 
dominant contribution to C\^'^ {jj). In [50] we have only taken C^ into account and 
therefore overestimated the effects of a fourth generation to the branching ratio of the decay 
h — )■ S7. Putting everything together we get 



A 



R 



SM3 



b—>-sj 



i?SM4 



(2.69) 



T/SM3 



ySM, 



SM4/-ye//3 



\SM4/-<; 



7 



lSM4/r^e//l 



^e//2 



^SM4^^ej,x ^ ^e;j.^ _ ^SM4(^e,ji ^ ^ej;. ^ 



SM4//-ye//l' 



ASM3C^-//3 _ XfMS^C^ffl + Cf^^] 



.(2.70) 



In [67] it was suggested to use the LO expression for 6 — > S7 at a low scale of /i = 3.22 GeV in 
order to reproduce the numerical value of the NLO expression. We have checked that Ab^s-y 
is quite insensitive to a variation of the scale between m,b and 3 GeV, so we use fi = m,b. 
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2.3.2 FCNC constraints with sensitivity to lepton mixing 

Next we also discuss FCNC processes that are sensitive to lepton mixing. In principle lepton 
mixing has to be investigated in the same manner as the quark mixing. For simplicity we have 
neglected lepton mixing in this paper. However, we will take into account the conservative 
bounds on Veu^, V^^^ and K1/4 given in [15] for the rare decay Bg -^ fi'^fi". In the ratio of the 
SM4 and SMS predictions for the branching ratios almost everything cancels out and one is 
left with the product of CKM elements and Inami-Lim functions 



BR{Bs -^ fi+fi' 



\SM4 



Ab.^MM ■■ ^^(^^ ^ l_l+f,-)SM3 



IXl'^'Yoixt) + Xl^'Yoix, 



\Xf^'Yo{xt)\' 



(2.71) 



with the Inami-Lim function 



^»W = Hl^ + ='(I^'"W). P.T2) 



Including also the leptonic contributions we have to make the following substitutions [67] in 
Eq. (I2TID 



Yo(xt) -^ Yoixt) - \U,,^rS{xt,x,,) , (2.73) 



|2 



Yo{xt>) -^ Yoixf) - lU^^r S{xt',x,,) , (2.74) 

where 5* is the box function given in Eq. (I2.4ip . x,y^ is given by the mass of the fourth 
neutrino and ?7^4 is the PMNS matrix element describing the mixing between the /i and the 
fourth neutrino. In [l5] the bound U^4 < 0.029 was derived. Using this information we find 
that leptonic contributions give at most a relative correction of 0.5 per mille, so we can safely 
neglect them. 

The branching ratio for B^ — ?■ fifi is not measured yet, HFAG quotes [114j (for the current 
experimental bound from TeVatron, see also [115] ) 

fir(5, ^ /i+/i") < 3.6 ■ 10~^ (2.75) 

In the SMS one expects a value of [HZ] 

Br{Bs ^ fi^fx^) = (3.2 ± 0.2) • 10^^ . (2.76) 

2.3.3 Allowed ranges for the A parameters 

Now we come to a crucial point: the fixing of the allowed ranges for the values of the 
different As. For our exploratory study - in comparison to a full fit that will be performed 
in future - we fix reasonable ranges for the As. Therefore we have to investigate theoretical 
and experimental errors. The FCNC quantities AMg, AM^ and b ^ sj are dominated 
by theoretical uncertainties. Currently, in particular the hadronic uncertainties are under 
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intense discussion, see e.g. |116] . Therefore, we use conservative estimates for the theoretical 
errors. 





Bound 


Ab, 


1±0.3 
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0±10° 


Ab. 


1±0.3 
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free 


M^k) 


1±0.5 


Im(A^) 


0±0.3 


Afe^s^ 


1±0.15 


^Bs^titi 


< 15 



Since we choose for the central values of our As the value one, all resulting allowed parameter 
points for VcKMi will be scattered around the SMS values by definition. For a future fit we 
will use As with the central value A = Q^^^/O^^^'^. 

This means in other words that we do not take into account some current deviations in flavor 
physics in our current analysis, we simply include them in our error band for the As. 



3 Putting things together — Constraints on the param- 
eter space 

In order to constrain the mixing with the fourth quark family we perform a scan through 
the parameter space of the model. To this end we use the exact parameterization of Vckm'^ 
described in Sec. 2.1, Eq. (12. 3p . For the tree-level bounds we use the central values and 
standard deviations as given in 12.2. 1( we allow for a variation at the 2a level. The restric- 
tive Peskin-Takeuchi parameters are allowed to vary at the 99% confidence level of [H], 
cp. Sec. 12.2.31 For the quark masses we use a hard lower limit of 280 GeV and allow for 
a maximal mass difference of 80 GeV as determined in Sec. 12.2.31 The lepton masses are 
chosen to be larger than 100 GeV with a maximal splitting of 140 GeV. For the FCNC we 
use the As given in Sec 12.3.31 

Then we generate a large number (0(10^^) ) of randomly distributed points in the 13 dimen- 
sional parameter spacq^. For each point we determine the value of CKM matrix elements, 
flavor and electroweak observables and check whether the various experimental bounds are 
passed (for details see [50]). 



3.1 Result for the mixing angles 

Let us for the moment ignore correlations among the various parameters and focus on the 
maximally allowed size of the mixing with the fourth generation. Table [1] shows the limits 
on the mixing angles 6*14, 6*24, and 6*34 — without and with the electroweak bounds. 
As already expected in Sec. 2, the Peskin-Takeuchi parameters impose strong constraints 
on the mixing of Standard Model fermions with the fourth generation. These numbers are 
comparable with the ones quoted in |S21 EZ] . The most dramatic effect is observed for the 
mixing angle ^34. The maximal size is roughly halfed by the virtue of the T parameter alone. 
So, already at this stage, one is able to conclude that a study of the flavor aspects of SM4 
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mt' , mb> , mi^ , m^^ , 6*12 , 6*23 , 6*13 , 



I,^34,<5l3 7<5l4,<524- 
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only tree-level 


with FCNC bounds 


with electroweak observables 


Ol4 


<0.07 


< 0.0535 


< 0.0535 


024 


<0.19 


< 0.145 


< 0.121 


^34 


<0.8 


<0.67 


<0.35 



Table 1: Maximal mixing of SM fermion generations with the fourth generation. The left 
column show the effect of tree-level bounds alone, for the central column FCNC bounds where 
added and the right-most column gives the limits including electroweak parameters. 
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Figure 3: Some correlations of the angles and phases. 

must not be decoupled from a simultaneous analysis of the electroweak sector. 

To illustrate the dependence of T on ^34 we also show the scatter plot for T versus ^34: 




-0,3 -0.2 



01 02 0.3 0.4 



Next, the correlations between the different angles are examined. The results are depicted 
in Fig. [3l Obviously, the maximal mixing angles given in Table [1] cannot be simultaneously 
realized; especially 614 and ^24 show a rather strong correlation and maximal ^24 is only 
possible for ^14 close to 0.018. Note e.g. the ^14 — ^34 correlation; simultaneous large mixing 
angles 6*34 and 6^14 are also disfavored. Indeed, this observation is rather natural, as Vta 
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includes a term 012013024^14^346*''". Hence, simultaneous large su and S34 would lead to a 
large modification of Vtd] B^ mixing would be sensitive to this and indeed proves to be the 
most restrictive of the mixing observables. Due to this observation we disfavor a strategy 
based on starting from fixed bounds on the mixing angles without taking the correlations 
into account. 

The large 'voids' in the ^13 — (5i4 plane can be traced to the effect of the direct limit on the 
phases due to the CKM angle 7. 

3.2 Results for the CKM elements 

Since the 3x3 unitarity fixes the values of the second and third row CKM elements rather 

precisely in the SM3, it is interesting to see the effect of the lifting of the unitarity constraint 

on Vcd, Vcs, Vc6, Vtrf, Vts and Vtb- In Figure H] we present the possible values for these CKM 

matrix elements in the complex plane. Note that a CKM matrix element itself is not a 

physical observable as it depends on phase conventions and CKM parameterization; one can, 

however, compare the values for the elements once the representation and phase convention 

is fixed. The plots correspond to the standard represention, cp. (12. ip . which is the limit of 

the Botella-Chau representation for zero mixing with a fourth family. 

The absolute value of the elements of the second row cannot change much with respect to 

the SM3; however, it is interesting to observe that the imaginary part of Vcd and Vcs can be 

increased by an order of magnitude. This might be potentially interesting for searches for 

CP violation in the charm sector. While Vcb does not have an imaginary part in the standard 

representation in SM3, a tiny imaginary part can be present in SM4. 

The absolute value of both, Vtd and Vts, can be modified (with respect to their SM3 value) 

by approximately a factor 2. More important, the imaginary part of V^s can be an order of 

magnitude larger than in the SM. Therefore, one can expect that the weak phases of processes 

involving Vts, s-g- Bs mixing, may experience large corrections. 

The absolute value of Vtb can be as low as 0.93. Without the constraints coming from 

oblique parameters this limit would be much lower — around 0.8. This again shows that the 

electroweak sector imposes strong limits on the flavor structure of SM4. 

This number is in particular interesting since it can be compared with direct determinations 

of Vtb from Single-top-production from TeVatron |117[ I118[ 1119] 

r,f^^'"° = 0.88 ±0.07. (3.77) 



3.3 New physics in Bg-mixing 

The results for the complex A^^ plane is particularly interesting since there might be some 
hints on new physics effects in the CP- violating phase of Bs mixing, see |106[ 1120] and the 
web-updates of [82]. In |106] a visualization of the combination of the mixing quantities AM^, 
AP^, all, which are known to NLO-QCD ^^ [ml [1221 [123] and of direct determinations 
of $s in the complex A-plane was suggested. Combining recent measurements |114l 1124] for 
the phase $s one obtains a deviation from the tiny SM-prediction |106] in the range of 2 to 
3(7: 

• HFAG: 2.2 a [TTl] . 

• CKMfitter: 2.1. ..2.5 a [T^IT^ . 
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Figure 4: Possible modifications of the SMS CKM matrix elements Vd, Vcs, Vcb Vtd, Vts and 
Vtb in the SM4 scenario. Depicted is the real part versus the imaginary part of the CKM 
element (in the standard representation). The crossed lines show the SMS value; Vcb is real 
by construction in the SMS. 

• UTfit: 2.9 a [HP] . 
The central values of these deviations cluster around 



$. 



-51°. 



(S.78) 



Very recently the DO collaboration announced a S.2cr deviation of a linear combination of af; 
and all jl^^l from the standard model prediction in |lU6j . This deviation also indicates a 
large negative value of ^g. 

As can be read off from the left picture of Figure O sizeable values for ^g can also be obtained 
in scenarios with additional fermions. Such large values for $s are not favored, but they 
are possible. An enhancement of ^s to large negative values by contributions of a fourth 
generation was first predicted in [ID] , by choosing the parameters of the fourth generations 
in such a way that other fiavor problems like the B — )■ Ktt puzzle are solved. Once the T 
parameter is implemented with full CKM dependence and used as an additional bound on the 
CKM elements, a large value of $s seems to be very unlikely and requires a significant fine 
tuning of the parameters, see the right picture of Figure [51 In that respect we differ slightly 
from the conclusion of e.g. ^\ |66l [671 1128] . where very large values for $s are allowed^ ^1. 
Here we expect new and considerably more precise data from TeVatron and LHC soon. If the 
central value stayed at the current position, the possibility would arise to find new physics 
that can not originate from an additional fermion family alone. 

It is interesting to study the mass dependence of the phase $s- Already in |128] it was 
noted that large phases clearly favor small t' (and b') masses and the largest phases require 
a value of m^ close to SOO GeV; this behaviour is also present in our analysis and it seems 



^^If we also described the problems in e.g. B — > Ktt by a fourth family, we also would exclude the points 
with $s close to zero and we would predict a sizeable phase - around —20° - but no points with e.g. —50° 
survive in our analysis. Because of hadronic uncertainties we did not include B — > Ktt in our analysis. 
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Figure 5: Complex A plane for Bf. mixing. The green shaded area corresponds roughly to 
the experiment at the Icr level |114] . The left panel shows the possible phase if one omits the 
T parameter, the right panel shows the impact of T on the allowed phase $s. 



that too large quark masses struggle to resolve the tension in the flavor sector should future 
experiments confirm e.g. a phase of the order of 30°. In Figj6]we show the distribution of the 
scatter plot for "light" (< 440 GeV) and "heavy" (> 440 GeV) i! masseS 

As a final remark, we would like to point out that the naively expected huge contributions 
to A^^ due to the heavy t' can (still) be veiled by the mechanism described in [50] • The new 
contributions in the SM4 fall into two classes. The first class contains the 'direct' effects 
of the new heavy ferminons, i.e., the contributions of diagrams with at least one t' in the 
loop. The second class is more subtle as it includes the 'indirect' effects of the SM4 scenario. 
These contributions arise from the breaking of the 3x3 unitarity; the standard model CKM 
elements have to be 'thinned' out to accomodate for the non-zero values of the fourth row 
and column matrix elements: this results in a modification of the SM-like contributions. 

These two sets of contributions are both sizeable (typically large enough to violate at least 
one experimantal bound), but they can cancel to a very large extent. Here is one example 
for the case of An which survived all our constraints: 



^12 


^13 


^23 


^14 


^24 


^34 




0.2275 


0.003409 


0.04036 


0.01701 


0.08392 


0.1457 




<^13 


^14 


<^24 


niti 


my 


mi^ 


my^ 


1.019 


0.91825 


1.0787 


385 GeV 


378 GeV 


602 GeV 


636 GeV 



This parameter set yields 

A« = 1 + 0.5379 



0.9016i -0.4196 + 0.4262i = 1.2 e 



-i23° 



(3.79) 



direct 



indirect 



One sees that each contribution is of the size of the Standard model result; however, direct 
and indirect contribution have opposite signs and cancel to a large extent. Note that in this 
case the 3-4 mixing is strong enough to allow for almost degenerate quark masses. 



-'^^Note that the points corresponding to "heavy" t' quarks are placed on top of the one corresponding to 
"hght" t's. A hght t' and a simuhancous smaU phase $s arc, of course, still possible. 
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Figure 6: Dependence of A^^ on the t' mass. The dark blue points correspond to masses 
below 440 GeV, the yellow (light grey) point correspond to masses heavier than 440 GeV. 
The red line indicates a phase $s of 30°. 

4 Taylor expansion of Vckm4: 



In [50] we gave a Wolfenstein-like expansion of the CKM matrix for four fermion generations, 
which allows for a first estimate of possible effects of the fourth generation. Since the inclusion 
of the electroweak oblique parameters led to tighter constraints for the mixing, one can now 
further improve this expansion. 

In the SMS the hierarchy of the mixing between the three quark families can conveniently 
be visualized via a Taylor expansion in the small CKM element Vus ~ 0.2255 = A: the 
Wolfenstein parameterization |129] . 
Following |130)^^l we define 



K. = Si2(l + 0(A«))=:A, 



(4.1) 
(4.2) 
(4.3) 



For the case of 4 generations one also needs the possible size, i.e. the power in A of the new 
CKM matrix elements. Using the results of the previous section we find: 

\Vub'\ < 0.0535 ^ 1.05A2 (1.05A2) , 



\Vcu\ < 0.123 ^ 0.54A^ ^ 2.38A^ 
1^46/1 <0.35^ 1.55A^ 



(2.8A2) , 
(3.0A^) . 



-'^^Note that due to historical reasons the element Vub is typically defined to be of order A'^, while it turned 
out that it is numerically of order A^ . 
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In brackets we show the results of our previous analysis in [50]. The biggest effect of the 
inclusion of the electroweak precision constrains was the reduction of the allowed mixing 
between the third and fourth family. The mixing between the first and the fourth family 
can still be considerably larger than the mixing between the first and the third family. This 
bound is still dominated by D-mixing. 



Defining the Vuf as 



one obtains 



Vub' =Si4e '^'^ =: X'^ixu - iyu) 



su = A \ xl^ + yf^ 



2 



^ci4 = l-A^^^^ + 0(A«) . (4.4) 

The parameters Xi4 and y^ are effectively smaller than or equal to 1 for all cases. 

• Let us further define the matrix element Vcb' via 

Vcb' = Cl4S24e-^^^^ =: (X24 - ty2A)X^ • (4.5) 

Comparison with Eq. (12. 3p then gives 

^ S24e-*'^^ = (X24 - ^2/24) A + O (A^) , 

^ C24 = 1 + ^ {-xl - yl,) X' + O (A^) . (4.6) 

• the last ingredient is the element Vtb''- 

Vtb' = C14C24S34 =: BX (4.7) 

and therefore 

sin(^34) = BX + h^ [Bxl, + Byl,) + O (A^) , 

cos(e34) = 1 - ^ + ^A^ {-B' - AB'xl, - ABY2,) + O (A^) . (4.8) 

Expanding the CKM4 matrix up to and including order A^, the matrix elements take the 
form : 

Kd = 1 - ^ + ^A^ (-4x?4 - 4yl -l)+0 (A^) , Ks = A , 

Vub = A{p - ifi)X^ , Vub' = (xu - iyu) A^ , 

(4.9) 
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Vcs =1 + -A^ (-x^4 - yl^ - 1) + 

+ ^A^ {-AA' - 2x^4 {yl - 1) - xi, - yt, + 2^24 - l) + O (A^) , 

Kb =^A^ , 

Kfe' =A^ (x24 - ^2/24) , (4.10) 

Kirf =A3 (A - Bxi4 - «%i4) + {-lAr^ -Ap + Bx24 + ^5y24) A' + O (A^) , 

l^i, = - AA^ - BX^ (X24 + ^2/24) + 2 A^ (^5' - ^2^24 - ^^24 + ^ " 25xi4 - 2z52/i4) + O (A^) 

K. =1 - ^ + l>^' (-4^' - 5' - 45^x^4 - ^BM) + O (A^) , 

Vtb'=BX, (4.11) 

V^'d =A^ (-X14 - iyu) + A^ (a;24 + ^^24) + 

+ -A^ {-2AB + (xi4 + iyu) [B^ + xj^ + yl^) + xm + ^^14) + O (A^) , 

Vt^s =A^ (-X24 - ^^24) + A^ {AB - xu - iyu) + - {B^ + l) A^ (x24 + ^2/24) + O (A^) , 

Vt^b = -BX- ^X^ [B [xl^ + yl^)) - AX" (^24 + ^2/24) + O (A^) , 

Vt'y =1 + ^A^ (-52 - xl^ - yl^ + 

+ ^A^ [-B" - 2B^xl^ - 2BY2A - 2x^4^2^ - ^24 - 4xL - vL - ^vl,) + O (A^) . 

(4.12) 

Naturally, this expansion cannot take into account correlations among the various CKM 
elements. However, the expansion is quite useful if one wants a rough estimate of the maximal 
size of a product of CKM elements determining the impact of the fourth generation on a 
certain process. 

5 Conclusion 

We have investigated the experimentally allowed parameter range for a hypothetical 4x4 
quark mixing matrix. Therefore we extended our previous study [SU] of the CKM like mixing 
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constraints of a fourth generation of quarks. 

Besides the tree- level determinations of the 3x3 CKM elements we also included the angle 
7 of the unitarity triangle, which turned out to be a rather severe bound for the phases of 
VcKM4:- Next we included the electroweak S, T, U parameters. Here we reproduced some 
of the results from [62], in particular we also excluded the three examples of very large 
mixing presented in [50]. In this paper we have included for the first time the full CKM 
dependence of the T and the U parameter. Doing so, we found that a mass degenerate 
fourth family of quarks is not excluded in that respect we differ e.g. from [Til El]- While 
degenerate quark masses can also arise if the lepton masses are adjusted accordingly, see 
Sect. 12.2.31 and [H], including the full CKM dependence allows for a greater 'flexibility' in 
the parameter space: e.g. only then a simultaneous separate degeneracy of leptons and quarks 
of the fourth generation would not be excluded — this may be of interest if one wants to 
invoke a symmetry to motivate the tiny mass splittings. In addition we found also that large 
values of the parameter U are not excluded a priori; only after applying the T parameter 
constraint we are left with small values of U. 

Concerning the FCNC constraints we studied K-, D-, B^-, -Bg-mixing and the decay b — ?> 57. 
In contrast to [50] we also included bounds to the rare decay Bg — )■ /x"*"//" and we improved 
our treatment of the QCD corrections to 6 — t- 57. It turned out that the naive bound for 
6 — )■ S7 used in [50] was too restrictive. 

Performing a scan over the whole parameter space of the SM4 we found that typically small 
mixing with the fourth family is favored, but still some sizeable deviations from the SM3 
results are not yet excluded. We demonstrated explicitly that e.g. effects of (9(100%) in Bg 
mixing are not excluded, yet. Concerning CP- violation in Bg mixing, we could have an almost 
arbitrarily large phase $s without violating the tree-level constraints. After switching on the 
T constraint (99 % CL of [2]) we could exclude values of the O{—50°) for the weak mixing 
phase <l>s; while values of O{—20°) can easily be obtained - this is still about two orders of 
magnitudes larger than the SM3 prediction ^g = 0.24° ±0.08° |106] . In that respect we differ 
slightly from [iQl [661 EH 1128] . If the real value of $« was —50%, this could not be achieved 
by an extension of the SMS, which consists only of an additional fermion familjo. Here new 
results for the 5-mixing observables from TeVatron and LHCb are very desireable. 
We found a minimal possible value for Vtb of 0.93 within the framework of the SM4, which 
can be compared with the result from single top production at the TeVatron |117[ I118[ 1119] : 
^TeVatron _ ggg _|_ Q_Q7_ jf jj^ naturc a Central value of Vtb = 0.88 was reahzed, this could not 
be explained by the SM4 alone. Here also more precise experimental data are desireable. 
In general we found a delicate interplay of electroweak and flavor observables, which strongly 
suggests that a separate treatment of the two sectors is not feasible. 

In our opinion the next steps to determine the allowed parameter space of the SM4 consist 
of i) performing a combined electroweak and CKM fit, ii) including lepton mixing and iii) 
including even more precision observables like e.g. Rb 01 B ^- K*ll. 
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